Clock Exercise

Q

How did you calculate your clients average clock drift rate?


After each successful request to the NTP server I divided the calculated clock offset by \(10 + 10*numfails\) where numfails is the number of failed attempts since the last successful attempt. Since a request to the server is made every 10 seconds this gives a good approximation for the drift per second (in microseconds). I kept an array of these values and calculated the average at each iteration. The final average was \(12.032miliseconds\)

Picking the Timeout

Question

what timeout did you pick to detect a failed interaction? What happens if the server’s response packet arrives after that timeout

I picked 10 seconds as the timeout. The resulting packet loss rate was \(4%\). When a timeout occurred the failure was counted and no other stats were calculated

while True:
        ...
        try:
                t3 = calc_time()
                data, address = client.recvfrom( 1024 )
                t0 = calc_time()
                succs += 1
        except Exception as ex:
                losses += 1
                logger.debug(f'timeout: {losses}')
                continue
        ...
        stat = {
                'offset':off,
                'RTT':rtt,
                'smoothed_offset':smoff,
                'smoothed_RTT':smrtt,
                'drop_rate':round(losses/(succs+losses), 2)*100,
                'current_system_time':now,
                'adjusted_system_time':adjusted,
                'current_drift':drift,
                'average_drift':sum(drifts)/len(drifts),
                'average_RTT':sum(rtts)/len(rtts),
        }
        logger.debug(stat)
        time.sleep(10)

Graphing the Clock Drift

I created this histogram of the clock drift per second (in miliseconds) for each successful interaction with the server. Note that there were 3200 interactions logged

As you can see most of the time the drift was positive, meaning that my machine’s clock was running faster than the NTP server’s. The histogram reflects a fairly normal distribution.

The scatter plot shows that the clock drift skews slightly to the right over time, but the width (range of the values) stays mostly constant. Adding in the plot of the average drift (calculated at each interaction) you can get a better picture of the overall trend. The average slowly increases over time which means that the client machine is getting out of sync with the NTP server at a faster rate.

Python Code for the NTP client

import socket
import struct
import sys
import time
import logging
import math
NTP_SERVER = "0.uk.pool.ntp.org"
TIME1970 = 2208988800

logger = logging.getLogger('rtt_and_offset')
logger.setLevel(logging.DEBUG)
handler = logging.FileHandler('rtt_offset.log')
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)


SOCKET_TIMEOUT = 10
NANOS = 1000000000

def calc_time():
    t = time.time()
    return (t//1, (t%1)*NANOS//1)

def diff(a, b):
    secs = b[0]-a[0]
    nanos = b[1]-a[1]
    nanos = (NANOS)*secs + nanos

    return nanos

def addnanos(it, nanos):
    newnanos, secs = it[1]+nanos, it[0]
    if newnanos >= NANOS:
        secs += 1 
        newnanos -= NANOS
    return (secs, newnanos)

stats, drifts, rtts = [], [], []
succs, losses, cur_fails = 0, 0, 0
while True:
    data = '\x1b' + 47 * '\0'
    data = data.encode('utf-8')
    client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    client.settimeout(SOCKET_TIMEOUT)
    client.sendto( data,( NTP_SERVER, 123 ))
    try:
        t3 = calc_time()
        data, address = client.recvfrom( 1024 )
        t0 = calc_time()
        succs += 1
    except Exception as ex:
        losses += 1
        logger.debug(f'timeout: {losses}')
        continue

    resp = struct.unpack( '!12I', data )

    reference = (resp[4]-TIME1970, resp[5]%NANOS)
    originate = (resp[6]-TIME1970, resp[7]%NANOS)
    receive   = (resp[8]-TIME1970, resp[9]%NANOS)
    transmit  = (resp[10]-TIME1970, resp[11]%NANOS)

    t1, t2 = transmit, receive

    off, rtt = (diff(t3,t2) - diff(t1,t0))/2, diff(t3, t0)
    drift = off/((10+(rtt/NANOS))*(losses-cur_fails+1))
    drifts.append(drift)
    cur_fails = losses

    rtts.append(rtt)

    stats.append((rtt, off))
    if len(stats) >= 8:
        stats = stats[1:]

    smrtt, smoff = min(stats, key=lambda st: st[0])

    now = calc_time()
    adjusted = addnanos(now, smoff)

    stat = {
            'offset':off,
            'RTT':rtt,
            'smoothed_offset':smoff,
            'smoothed_RTT':smrtt,
            'drop_rate':round(losses/(succs+losses), 2)*100,
            'current_system_time':now,
            'adjusted_system_time':adjusted,
            'current_drift':drift,
            'average_drift':sum(drifts)/len(drifts),
            'average_RTT':sum(rtts)/len(rtts),
    }

    logger.debug(stat)

    time.sleep(10)



Python Code for Parsing the Log and Producing the Graphs

import re
import pandas
import numpy
import plotly.express as px
import plotly.graph_objects as go
from types import SimpleNamespace

with open('rtt_offset.log') as log:
    data = re.findall('{.*?}', log.read())

stats = [eval(it) for it in data]

avg_RTT = stats[-1]['average_RTT']/1000000000
packet_loss_rate = stats[-1]['drop_rate']
avg_drift_in_milis = stats[-1]['average_drift']/1000000


for stat in stats:
    stat['current_drift'] /= 1000000
    stat['average_drift'] /= 1000000

frame = pandas.DataFrame(stats)

fig = px.histogram(frame, x='current_drift', 
        marginal = 'violin',
        title='Histogram of Clock Drift Per Second', 
        labels={'current_drift':'drift_in_miliseconds_per_second', 'y':'percent of records'}, 
        opacity=0.7, 
        color_discrete_sequence=['indianred'],
        hover_data=frame.columns)
fig.update_layout(xaxis_title="Drift in miliseconds/sec", yaxis_title="Count")
fig.write_html('fig1.html')

fig2 = px.scatter(frame, x='current_drift')
fig2.add_trace(go.Scattergl(
    x=frame.average_drift, 
    mode='markers', 
    name='average drift at each interaction',
     marker=dict(
        size=10,
        color=numpy.random.randn(1000), #set color equal to a variable
        colorscale='Viridis', # one of plotly colorscales
        line_width=1
    )
))
fig2.update_layout(title='Scatterplot of Clock Drift', 
        xaxis_title="Drift in miliseconds/sec", 
        yaxis_title="Count")
fig2.show()
fig2.write_html('fig2.html')